Skip to content

[Model] Deprecate the score task (this will not affect users). #37537

Merged
noooop merged 7 commits intovllm-project:mainfrom
noooop:deprecate_score
Mar 20, 2026
Merged

[Model] Deprecate the score task (this will not affect users). #37537
noooop merged 7 commits intovllm-project:mainfrom
noooop:deprecate_score

Conversation

@noooop
Copy link
Copy Markdown
Collaborator

@noooop noooop commented Mar 19, 2026 •
edited by github-actions bot
Loading

Purpose

Cross-encoder (aka reranker) models are a subset of classification models that accept two prompts as input and output num_labels equal to 1.

In the engine, we will no longer differentiate between classify and score. So we can deprecate the score task.

This is part of deprecate support for multi-task.

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@noooop
init
e0d8d48 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
gemini-code-assist[bot]
gemini-code-assist bot reviewed Mar 19, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request successfully deprecates the 'score' task, replacing it with 'classify' where appropriate and removing related logic and comments. The changes are consistent across tests, model configuration, entrypoints, and model executor components. The refactoring of activation functions and renaming of variables like pooled_data to embeddings or logits enhances code clarity and maintainability. No critical or high-severity issues were found in the changes.

@noooop
refine
fb987a9 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
@noooop noooop marked this pull request as ready for review March 19, 2026 09:05
@noooop noooop added ready ONLY add when PR is ready to merge/full CI is needed and removed ready ONLY add when PR is ready to merge/full CI is needed labels Mar 19, 2026
DarkLight1337
DarkLight1337 reviewed Mar 19, 2026
View reviewed changes
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Mar 19, 2026

Hi @noooop, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

@noooop
refine
d7ad6d1 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
@noooop noooop added ready ONLY add when PR is ready to merge/full CI is needed and removed ready ONLY add when PR is ready to merge/full CI is needed labels Mar 19, 2026
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 20, 2026
@DarkLight1337 DarkLight1337 disabled auto-merge March 20, 2026 02:41
@noooop noooop enabled auto-merge (squash) March 20, 2026 03:03
@noooop noooop merged commit ed359c4 into vllm-project:main Mar 20, 2026
74 checks passed
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Mar 21, 2026
Revert "[Model] Deprecate the score task (this will not affect users). (
63e2539 
vllm-project#37537)"

This reverts commit ed359c4.
@zhewenl zhewenl mentioned this pull request Mar 21, 2026
Closed
chooper26 pushed a commit to intellistream/vllm-hust that referenced this pull request Mar 21, 2026
@noooop @ShuhaoZhangTony
[Model] Deprecate the score task (this will not affect users). (vllm-…
5049582 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
@noooop noooop deleted the deprecate_score branch March 21, 2026 04:57
@robertgshaw2-redhat
Copy link
Copy Markdown
Collaborator

robertgshaw2-redhat commented Mar 21, 2026

@noooop - this broke CI
@DarkLight1337 - please ensure we run relevant tests

@noooop
Copy link
Copy Markdown
Collaborator Author

noooop commented Mar 22, 2026 •
edited
Loading

@noooop - this broke CI
@DarkLight1337 - please ensure we run relevant tests

test below passed successfully when landing.

https://buildkite.com/vllm/ci/builds/56995/steps/canvas?jid=019d058c-c6a8-44d4-90b5-adbb3c3bfe7f&tab=output

let's rerun this test.

https://buildkite.com/vllm/ci/builds/57220/steps/canvas?sid=019d09f2-b0ab-4b55-abae-5ffcd500c440&tab=output

https://buildkite.com/vllm/ci/builds/57231/steps/canvas?sid=019d0a4a-8d65-4473-94ce-9944c14d21be&tab=output

https://buildkite.com/vllm/ci/builds/57288/steps/canvas?sid=019d0be7-c922-4bac-9e36-1dae8d24be1a&tab=output

https://buildkite.com/vllm/ci/builds/57302/steps/canvas?sid=019d0c41-4cef-4034-a5ea-8a40b9a97275&tab=output

All these tests passed.

I also cannot reproduce this error locally. That's too strange.

@noooop
Copy link
Copy Markdown
Collaborator Author

noooop commented Mar 22, 2026

image

@noooop
Copy link
Copy Markdown
Collaborator Author

noooop commented Mar 22, 2026

image

@noooop
Copy link
Copy Markdown
Collaborator Author

noooop commented Mar 22, 2026

Perhaps it was caused by #37303, and it should be fixable by #37775.

cc @robertgshaw2-redhat @AndreasKaratzas

@AndreasKaratzas
Copy link
Copy Markdown
Collaborator

AndreasKaratzas commented Mar 22, 2026 •
edited
Loading

@noooop i can open it, i just want also #37774 to be merged so that amd ci is evaluated as well. Can we merge 37774 first?

This was referenced Mar 23, 2026
Draft
Merged
SouthWest7 pushed a commit to SouthWest7/vllm that referenced this pull request Mar 27, 2026
@noooop
[Model] Deprecate the score task (this will not affect users). (vllm-…
07804ef 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
khairulkabir1661 pushed a commit to khairulkabir1661/vllm that referenced this pull request Mar 27, 2026
@noooop @khairulkabir1661
[Model] Deprecate the score task (this will not affect users). (vllm-…
23dc6f8 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Monishver11 pushed a commit to Monishver11/vllm that referenced this pull request Mar 27, 2026
@noooop @Monishver11
[Model] Deprecate the score task (this will not affect users). (vllm-…
5e5b26f 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
nithinvc pushed a commit to nithinvc/vllm that referenced this pull request Mar 27, 2026
@noooop
[Model] Deprecate the score task (this will not affect users). (vllm-…
516ba0e 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: Nithin Chalapathi <nithin.ch10@gmail.com>
JiantaoXu pushed a commit to JiantaoXu/vllm that referenced this pull request Mar 28, 2026
@noooop @JiantaoXu
[Model] Deprecate the score task (this will not affect users). (vllm-…
240e22d 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
vrdn-23 pushed a commit to vrdn-23/vllm that referenced this pull request Mar 30, 2026
@noooop @vrdn-23
[Model] Deprecate the score task (this will not affect users). (vllm-…
8f66714 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
EricccYang pushed a commit to EricccYang/vllm that referenced this pull request Apr 1, 2026
@noooop @EricccYang
[Model] Deprecate the score task (this will not affect users). (vllm-…
e94e98d 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: EricccYang <yangyang4991@gmail.com>
@ppraneth ppraneth mentioned this pull request Apr 3, 2026
Open
4 tasks
liuchenbing2026 pushed a commit to liuchenbing2026/vllm that referenced this pull request Apr 4, 2026
@noooop @liuchenbing
[Model] Deprecate the score task (this will not affect users). (vllm-…
a12faaa 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
@rafvasq rafvasq mentioned this pull request Apr 6, 2026
Merged
5 tasks
big-yellow-duck pushed a commit to EmbeddedLLM/vllm that referenced this pull request Apr 8, 2026
@noooop @big-yellow-duck
[Model] Deprecate the score task (this will not affect users). (vllm-…
a22b592 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
tjohnson31415 pushed a commit to vllm-project/vllm-spyre that referenced this pull request Apr 8, 2026
@rafvasq
chore: add support for vllm v0.19.0 (#898)
3e8e966 
<!-- markdownlint-disable -->

## Description

Add support for vLLM v0.19.0

- bump vllm versions
- Inputs reorganization
([#35182](vllm-project/vllm#35182))
- `get_cross_encoder_act_fn` merged into `get_act_fn`
([#37537](vllm-project/vllm#37537))
- `RequestStatus.WAITING_FOR_FSM` renamed to
`WAITING_FOR_STRUCTURED_OUTPUT_GRAMMAR`
([#38048](vllm-project/vllm#38048))
- `prompt_token_ids_cpu` arg in PoolingMetadata
([#38139](vllm-project/vllm#38139))

## Related Issues

<!-- Link related issues, e.g., `Fixes #` or `Relates to #456` -->


## Checklist

- [x] I have read the [contributing
guidelines](https://docs.vllm.ai/projects/spyre/en/latest/contributing)
- [x] My code follows the project's code style (run `bash format.sh`)
- [x] I have added tests for my changes (if applicable)
- [ ] I have updated the documentation (if applicable)
- [x] My commits include a `Signed-off-by:` line (DCO compliance)

---------

Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
mtparet pushed a commit to blackfuel-ai/vllm that referenced this pull request Apr 9, 2026
@noooop @mtparet
[Model] Deprecate the score task (this will not affect users). (vllm-…
79ee2a7 
…project#37537)

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DarkLight1337 DarkLight1337 DarkLight1337 approved these changes

@njhill njhill Awaiting requested review from njhill njhill is a code owner

@aarnphm aarnphm Awaiting requested review from aarnphm aarnphm is a code owner

@chaunceyjiang chaunceyjiang Awaiting requested review from chaunceyjiang chaunceyjiang is a code owner

@russellb russellb Awaiting requested review from russellb russellb is a code owner

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon WoosukKwon is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth is a code owner

@houseroad houseroad Awaiting requested review from houseroad houseroad is a code owner

@hmellor hmellor Awaiting requested review from hmellor hmellor is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 is a code owner

@ProExpertProg ProExpertProg Awaiting requested review from ProExpertProg ProExpertProg is a code owner

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation frontend ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@noooop @robertgshaw2-redhat @AndreasKaratzas @DarkLight1337